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Abstract. A multi-source quickest detection problem is considered. Assume there are two independent Poisson 
processes X 1 and X 2 with disorder times 81 and 82, respectively; that is, the intensities of X 1 and X 2 change at random 
unobservable times 81 and 82, respectively. 81 and 82 are independent of each other and are exponentially distributed. 
Define 8 = 8 1 A 8 2 = min{0i,6» 2 } . For any stopping time r that is measurable with respect to the nitration generated by 
the observations define a penalty function of the form 

R T = P( T <8) + cE [(t - 8)+] , 

where c > and (r — 0)+ is the positive part of r — 8. It is of interest to find a stopping time r that minimizes the above 
performance index. This performance criterion can be useful for example in the following scenario: There are two assembly 
lines that produce products A and B, respectively. Assume that the malfunctioning (disorder) of the machines producing A 
and B are independent events. Later, the products A and B are to be put together to obtain another product C. A product 
manager who is worried about the quality of C will want to detect the minimum of the disorder times (as accurately as 
possible) in the assembly lines producing A and B. Another problem to which we can apply our framework is the internet 
surveillance problem: A router receives data from, say, n channels. The channels are independent and the disorder times 
of channels are 61, ■■■ ,8 n . The router is said to be under attack at 8 = 61 A ■ • • A 6 n . The administrator of the router is 
interested in detecting 8 as quickly as possible. 

Since both observations X 1 and X 2 reveal information about the disorder time 8, even this simple problem is more 
involved than solving the disorder problems for X 1 and X 2 separately. This problem is formulated in terms of a three 
dimensional sufficient statistic, and the corresponding optimal stopping problem is examined. The solution is characterized 
by iterating a suitable functional operator. 

AMS subject classifications. 62L10, 62L15, 62C10, 60G40. 
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1. Introduction. Consider two independent Poisson processes X' 1 — {X\ : t > 0} i € {1,2} with 
the same arrival rate [3. At some random unobservable times 6\ and #2, with distributions 

P(9. 1 =0)=tt 1 P(6>i > t) = (1 - 7r 4 )e~ At for t > 0, (1.1) 

the arrival rates of the Poisson processes X 1 and X 2 change from (3 to a, respectively, i.e., 

X\- [ hi(s)ds, t>0, i = l,2, (1.2) 
Jo 

are martingales, in which 

hi(t) = [/31 {s <e l} +«!{«>««}], t>0,t = l,2. (1.3) 

Here a and /3 are known positive constants. We seek a stopping rule r that detects the instant 9 = 9\ A #2 
of the first regime change as accurately as possible given the past and the present observations of the 
processes X 1 and X 2 . More precisely, we wish to choose a stopping time r of the history of the processes 
X 1 and X 2 that minimizes the following penalty function 

R T = P(t < 6>) + cE [(r - $)+] . (1.4) 
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The first term in p.4[) penalizes the frequency of false alarms, and the second term penalizes the detection 
delay. The disorder time demarcates two regimes, and in each of these regimes the decision maker uses 
distinctly different strategics. Therefore, it is in the decision maker's interest to detect the disorder time 
as accurately as possible from its observations. Here, we are solving the case when a decision maker has 
two identical and independent sources to process. In the Section [9] we discuss how our analysis can be 
extended to non-identical sources. 

Quickest detection problems arise in a variety of applications such as seismology, machine monitoring, 
finance, health, and surveillance, among others (see e.g. PQ, [10], [7], [H] and [H]). Because Poisson 
processes are often used to model abrupt changes, Poisson disorder problems have potential applications 
e.g. to the effective control and prevention of infectious diseases, quickest detection of quality and 
reliability problems in industrial processes, and surveillance of Internet traffic to protect network servers 
from the attacks of malicious users. This is because the number of patients infected, number of defected 
items produced and number of packets arriving at a network node are usually modeled by Poisson 
processes. In these examples the disorder time corresponds to the time when an outbreak occurs, when 
a machine in an assembly line breaks down or when a router is under attack, respectively. The multi- 
source quickest detection problem considered here can be applied to tackle these problems when there are 
multiple sources of information. For example in the monitoring of industrial processes the minimum of 
disorder times represents the first time when one of many assembly lines in a plant breaks down during 
the production of a certain type of item. Let us be more specific: Assume that there are two assembly 
lines that produce products A and B, respectively. Assume also that the malfunctioning (disorder) of the 
machines producing A and B are independent events. Later, the products A and B are to be put together 
to obtain another product C. A product manager who is worried about the quality of C will want to 
detect the minimum of the disorder times (as accurately as possible) in the assembly lines producing A 
and B. The performance function (jl.4|) is an appropriate choice because the product manager will worry 
about the quality of the end product C, not of the individual pieces seperately. Another problem to 
which we can apply our framework is the internet surveillance problem: A router receives data from, say, 
n channels. The channels are independent and the disorder times of channels are 8±, ■ ■ ■ ,9 n . The router 
is said to be under attack at 6 = 6\ A • • • A 9 n . The administrator of the router is interested in detecting 
9 as quickly as possible. 

The one dimensional Poisson disorder problem, i.e., the problem of detecting 9\ as accurately as possible 
given the observations from the Poisson process X 1 has recently been solved (see [2J, [3J and the references 
therein). The two-dimensional disorder problem we have introduced cannot be reduced to solving the 
corresponding one-dimensional disorder problems since both X 1 and X 2 reveal some information about 9 
whenever these processes jump. That is, if we take the minimum of the optimal stopping times that solve 
the one dimensional Poisson disorder problems, then we obtain a stopping time that is a sub-optimal 
solution to (|1.4|) (see Remark |4.1|) . 

We will show that the quickest detection problem of (|1.4[) can be reduced to an optimal stopping prob- 
lem for a three-dimensional piece-wise deterministic Markov process. Continuous-time Markov optimal 
stopping problems are typically solved by formulating them as free boundary problems associated with 
the infinitesimal generator of the Markov process. In this case, however the infinitesimal generator con- 
tains differential delay operators. Solving free boundary problems involving differential delay operators 
is a challenge even in the one dimensional case and the smooth fit principle is expected to fail (see [2], 
[3J and the references therein). Instead as in [4] and [6] we work with an integral operator, iteration of 
which generates a monotonically increasing sequence of functions converging exponentially to the value 
function of the optimal stopping problem. That is, using the integral operator we reduce the problem 
to a sequence of deterministic optimization problems. This approach provides a new numerical method 
for calculating and characterizing the value function and the continuation region in addition to providing 
information about the shape and the location of the optimal continuation region. Using the structure 
of the paths of the piece-wise deterministic Markov process we also provide a non-trivial bound on the 
optimal stopping time which can be used to obtain approximate stopping strategies. 

The remainder of this paper is organized as follows. In Sections [5J and [31 we restate the problem 
of interest under a suitable reference measure Pq that is equivalent to P. Working under the reference 



Quickest Detection of a Minimum of Two Poisson Disorder Times 



3 



measure Po reduces the computations considerably, since under this measure the observations X 1 and X 2 
are simple Poisson processes that are independent of the disorder times. Here we show that the quickest 
detection problem reduces to solving an optimal stopping problem for a three-dimensional statistic. 
In Section 21 we analyze the path behavior of this sufficient statistic. In Section [5j we provide a tight 
upper bound on the continuation region of the optimal stopping problem, which can be used to determine 
approximate detection rules besides helping us to determine the location and the shape of the continuation 
region. Here, we also show that the smallest optimal stopping time of the problem under consideration has 
finite expectation. In Section [6l we convert the optimal stopping problem into sequences of deterministic 
optimal stopping problems using a suitably defined integral operator. In Section [71 we construct optimal 
stopping times from sequences of stopping (alarm) times that sound before the processes X 1 and X 2 
jump a certain number of times. In Section [8] we discuss the structure of the optimal stopping regions. 
And finally, we discuss how to extend our approach to the case with more than two sources, and to the 
case when the jump sizes are random and the jump size distribution changes at the time of disorder. 

2. Problem Description. Let us start with a probability space (CI, J 7 , Po) that hosts two inde- 
pendent Poisson processes X 1 and X 2 , both of which have rate j3, as well as two independent random 
variables Q\ and #2 independent of the Poisson processes with distributions 

P (^ = 0) = 7T 4 and P„(0i > t) = (1 - TTiJe-***, (2.1) 

for < t < co, i G {1, 2} and for some known constants Ki € [0, 1) and A > for i e {1,2}. We denote 
by F = {J r t}o<t<oo the filtration generated by X 1 and X 2 , i.e., Tt = <r(Xl,X 2 ,0 < s < t), and denote 
by G = {Gt}o<t<oo the initial enlargement of F by 6 1 and 9 2 , i.e., Q t = a(9i,9 2 ,X 2 ,X 2 :0<s<t). The 
processes X 1 and X 2 satisfy (|1.2[) under a new probability measure P, which is characterized by 

d¥ 

dPo" 

where 

Z\ 4 exp (J* log (^j^) dX\ - J\hi(s) - fids) , (2.3) 

for t > and i <G {1, 2} are exponential martingales (see e.g. [5]). Under this new probability measure P 
of (|2.2[) . 9\ and 02 have the same distribution as they have under the measure Po, i.e., their distribution is 
given by (jl.ip . This holds because 9\ and #2 are C/o-measurable and <flP/<flPo| g = 1, i.e., P and Po coincide 
on Q . Under the new probability measure P the processes X 1 and X 2 have measurable intensities hi and 
I12 respectively. That is to say that (|1.2[) holds. In other words, the probability space (f2, T, P) describes 
the model posited in (jl.ip and (|1.2p . Now, our problem is to find a quickest detection rule for the disorder 
times 9\f\92i which is adapted to the history T generated by the observed processes X 1 and X 2 because 
the complete information (concerning 9\ and 62) embodied in Q is not available. We will achieve our goal 
by finding an T stopping time that minimizes (|1.4|) . 

In terms of the exponential likelihood processes 

Ll=(j) ' exp(-(a -/?)*), t>0,ie {1,2}, (2.4) 

we can write 

1} 

h 9i 



A ry A rr\ r?2 

— 6 t — Z t Z t , 



(2.2) 



Let us introduce the posterior probability process 



H t 4 P ( B < t \T t ) = E ° [gftg^llgj , (2.6) 
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where the second equality follows from the Bayes formula (see e.g. [5]). Then it follows that from (|2.5 
and RIM that 



(1 - 7r)e _2At 
1 — ITj = f — , — t— , where 



(2.7) 



7r^l-(l-7ri)(l-7r 2 ). 
Let us now introduce the odds-ratio process 



l-n, 



■, < t < oo. 



Then observe from ((23)1 and (j2~7) that 

E [Z t l {e < t] \F t ] = (l-7T)e- At $ t , 

t > 0. Now, we will write the penalty function of (|1.4|) in terms of the odds-ratio process. 

" />OC I />00 

E[(r-0)+]=E / l {T>t }l { e<t}dt = E [l {r>t] E [Z t l {e < t] \F t ]] dt 

= (1 - tt)E / e~ 2At $ t dt. 
Jo 

Since {r < 0} € £/# we can write 

P(t < 0) - E [Z e l {r<e} ] - Po(r < 0) = (1 - tt) f 1 - AE J , 



~ 2Xt dt 



(2.8) 



(2.9) 



(2.10) 



(2.11) 



(2.12) 



where the second equality follows since Zg = 1 almost surely under P . Using (|2.11[) and (|2. 12[) we can 
write the penalty function as 



i?r(7ri,7r 2 ) = 1 -7r + c(l - tt)E 



A 



e -2At | _ , 



On the other hand the following lemma obtains a representation for the odds ratio process $. 
Lemma 2.1. Let its denote 



-,A/ 



1 - 7Tj 



7" 1 

i * It" 



< TO 



< w 



/or i > and i € {1, 2}. TTien we can write t/ie odds-ratio process $ as 

$t = $1 + $ 2 + $J$ 2 , <>0 



(2.13) 



(2.14) 



(2.15) 



Proof. From (pOO]) 



2At 



(1-7T) 



„2Xt 



E [Z t l {8 < t} |^ t ] , 



(1-tt) 



P o (0i > <)E 



1 ^* I T 1 



J 0i 



En 



^ I 1 

■{6i<t}7rK4 

1 {e2<t}T2~Kt 

^ 2 



+ P(6» 2 > t)Eo 



1 {e2<t}T2"Kt 



(2.16) 
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The second equality follows from (|2.2[) , (|2.5[) and the independence of the sigma algebras J-\ and T^. 
Now the claim follows from (|2~Tjl . ([23]) and (|2~T4| . □ 



1 - 7T , 



Using the fact that the likelihood ratio process L l is the unique solution of the equation 

dL\ = [{a/ 13) - l]L\_{dX i - adt), 4 = 1, 
(see e.g. [13]) and by means of the chain-rule we obtain 

d$i = (\+{\-a + P)$i)dt+[(a/0)-l}$idXi, & n = 

for t > and i € {1, 2} (see 0]). If we let 

$+ = *J + $?, $ t x = t > 0, 

then using a change of variable formula for jump processes gives 

d$? = [A$+ + 2a^]dt + {a/0) - l)$ t x d(*t + X 2 ), 
d4>+ = [2A + a$+]di + ((a/ (3) - l)[& t dXi + <S> 2 t dX?} 



(2.17) 



(2.18) 



(2.19) 



(2.20) 



with $q = 7Ti7r 2 /[(l - 7Ti)(l - 7r 2 )], and = 7Ti/(l - tti) +7r 2 /(l - 7r 2 ), where a = A - a + /?. Note that 
Xt = X\ + Xf 7 t > 0, is a Poisson process with rate 2/3 under Po- 

It is clear from ([2~T8) and ([2T2D|) that 



T = ($ x ,$+,$^ 



(2.21) 



is a piece- wise deterministic Markov process; therefore the original change detection problem with penalty 
function (|1.4[) has been reformulated as (|2.13[) and (|2.18[) - (|2.21[) . which is an optimal stopping problem for 
a two dimensional Markov process driven by three dimensional piecewise-deterministic Markov process. 

We will denote by A the infinitesimal generator of T. Its action on a smooth test function / : — > K 
is given by 



[Af] 



^ 0+, 1 ) = D^, f(r , <j>+, ^)[X4>+ + 2a^+] + D^fir , </>+, ^)[2A + a^+] 



+ ^ 0l /(0 x ^+,0 1 )[A + a0 1 ]+ / 3 
+ 



/If. 



^A 1 I - /(0 X ,0 + ,0 1 ) 



x A+ 



(2.22) 



Let us denote 



= {(x,y)eR 2 + :y>2yG} and 



(2.23) 



83 A 



^ {(x,y,z) & R 3 + : y >2^,y > z}. 



(2.24) 



Now, for every 
solutions of 



b 1 ) e B+, let us denote denote by x(t, X ), y(t,4> + ) and z(i, (/j 1 ), (6l, the 



eft 
d 

d 
di 



x(t, <f> x ) = [Ay(t, cj)+) + 2ax(t, X )]dt, x(0, X ) = </» x , 
0+) = [2A + ay(t, <f>+)]dt, y(0, 0+) - 0+, 



(2.25) 



z(t, (j) 1 ) = [A + as(i, x )]dt, z(0, (j) 1 ) = (j) 1 . 
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The solutions of (|2.25p , when a ^ 0, are explicitly given by 

A 2 



x(t,<f> x ) = ¥r + e 2at 



+ e 2at (l-e- at )- 



2A 
a 



y(t, ^ 



z{t, fa 1 ) 



2A 



it A+ 



2A 



(2.26) 



Otherwise, x{t,cj) x ) = 4> x + At0 x + A 2 i 2 , y(t,cj) + ) = <j)+ + 2Xt, and z(t,fa) = + At. Note that the 
solution (x, y, z) of the system of equations in (12.251) satisfy the semi-group property, i.e., for every s,(eM 

x{t + s,(j) ) = x(s,x(t,4> )), y(t + s,fa) = y(s,y(t, fa)), and z(t + s, fa) = z(s, z(t, fa)). (2.27) 

Note from (JHHD, ([^2H|l and (|2~2"6"1) that 

x(t - a n , ** ), $+ = y(t - C7„, $+ ), ®\ = z(t - a n , J <r„ < i < <r n+1) n e N, (2.28) 



and 



= — <I> X 

+1 0*v„+l-> 



$l n+1 =|*^i 1 {xi B+1 ^i B+1 _}' and 



/3 



1 "n+l^ <T n + l--' 



+ 



/3 



(3 I CT " +I 



(2.29) 



Here, for any function h, h(t—) = lim s | t h(t). Note that an observer watching T is able to tell whenever 
the processes X 1 and X 2 jump (see (|2.18[) . (|2.20p ). i.e., the filtration generated by T is the same as F. 

3. An Optimal Stopping Problem. Let us denote the set of F stopping times by S. The value 
function of the quickest detection problem 



U(ni,Tr 2 ) = inf i? T (7ri,7r 2 ) 
res 



can be written as 



C/(7r 1 ,^ 2 ) = (l-7r) 



1 +cV 



7Tl7T2 771+772 — 27T17T2 TT\ 



1 — 7T 1 — 7T 

where V is the value function of the optimal stopping problem 



1 — 7Tl 



Vi^^+A 1 ) = inf E^ X ' 0+ ' 0l) 



Jo 



(3.1) 



(3.2) 



(3.3) 



in which (4> x , <p + , fa ) 6 Dj_, and h(x,y) = x + y — A/c. Here Eg ' is the expectation under Po 

given that $q = X , $J = </>+, $q = ^ 

It is clear from (|3.3[) that for both optimal stopping problems it is not optimal to stop before (4> t x , $j ), 
t > 0, leaves the advantageous region defined by 



Let us also denote 



C O ^{(0 X ,0+) em 2 + :fa +0+ < A/c}. 



C 4 , 0+, 1 ) eB^:^+^ + < A/c}. 



(3.4) 



(3.5) 



Also note that the only reason not to stop at the time of the first exit from the region Co is the prospect 
of ($* , $j ), t > returning to Co at a future time. 

Remark 3.1. It is reasonable to question our choice of statisitc, since it is clear that (<!>(, $ 2 ) t > 
contains all the information X has to offer. Our choice (T t ) t >o, which is defined in i2.21\) . is motivated 
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by the mere desire of having a concave value function U and a convex optimal stopping region. The 
concavity is due to the linearity of the function h ( see Lemma \6.2\ and its proof along with Lemmas \6.1\ 
and \6. and Theorem \6.1\) . 



If we had chosen to work with {$\, $ 2 )t>o, then the relevant optimal stopping problem becomes 



W(a, b) 4 inf E a ' b) 



(3.6) 



in which 

h(x,y)=x + y + xy, (x,y) eM. 2 + . (3.7) 

Sinceh(-,-) is non-linear the concavity of the value function, i.e. W(-,-) is not concave. In fact, W(x,y) — 
V(xy,x +y,x). The function V is concave but W is not. So there is a trade off between concavity and 
the dimension of the statistic to be used. 

In what follows, for the sake of the simplicity of notation, when the meaning is clear, we will drop the 
superscripts of the expectation operators E ^ ' ^ ' ^ \ 

4. Sample Paths of \& = ($ x , <£>+). It is illustrative to look at the sample paths of the sufficient 
statistic vp, to understand the nature of the problem. Indeed, this way, for a certain parameter range, we 
will be able to identify the optimal stopping time without any further analysis. 

From (|2.26p . we see that, if a > 0, then the paths of the processes <& x and <I> + increase between the 
jumps, and otherwise the paths of the processes <I> X and <I> + mean-revert to the levels 2A 2 /a 2 and — 2A/a 
respectively. Also observe that $ x , <f> + increase (decrease) with a jump if a > j3 ((3 > a). See (|2.20ll . 

Case I: a> (3, a > 0. The following theorem follows from the description of the behavior of the paths 
above. 

Theorem 4.1. If a > (3 and a > 0, then the stopping rule 

r = inf{i > : $ f x + $^ > A/c} (4.1) 

is optimal for (3. Sty . 

Proof. Under the hypothesis of the theorem and whenever a path of ($ x , $ + ) leaves Co it never returns. 

□ 

In Section [5l we will identify another case (another range of parameters) in which the advantageous 
region Co is the optimal continuation region and the stopping time tq is optimal (see Cases II-b-i-1 and 
II-b-ii-1). 

Remark 4.1. Let Ki = inf{£ > : $J > A/c}. If a > /3 and a > 0, then Ki is the optimal stopping 
time for the one dimensional disorder problem with disorder time 9i Let us define k — k\ A Ki. 

Since with this choice of parameters $ x + > A/c, it follows that tq < k almost surely. Therefore, if 
we follow the rule dictated by the stopping time k, then we pay an extra penalty for detection delay. This 
example illustrates that solving the two one dimensional quickest detection problems separately in order 
to minimize the penalty function of \l-4ty is suboptimal. 

In what follows, we will consider the remaining cases: a > (3 and a < 0; a < [3. 



5. Construction of a Bound on the Continuation Region. In this section the purpose is to 
show that the continuation region of (|3.3[) is bounded. The construction of upper bounds is carried out 
in the next two theorems. These upper bounds are tight as the next theorem shows and might be used to 
construct useful approximations to the two optimal stopping times solving the problems defined in (|3.3|) . 
We will carry out the analysis for a = A — a + [3 ^ 0. A similar analysis for this case can be similarly 
performed. As a result of Theorem 15.31 we are also able to conclude that the (smallest) optimal stopping 
time has a finite expectation. 
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The first two theorems in this section assume that an optimal stopping time of (|3.3[) exists and in 
particular the stopping time 

t* (a, b, c) 4 inf{i > : V(T t ) = 0, T = (a, b, c)}, (5.1) 

is optimal. In Section [3 we will verify that this assumption holds. We will denote by 

r = {(a,b,c) £ M 3 + : v(a,b,c) = 0}, C = K\_ - V, (5.2) 

the optimal stopping region and optimal continuation region of (|3.3p respectively. 

Theorem 5.1. In this theorem the standing assumption is that a> (3 and that a < (Case II). 

(Case Il-a) Let us further assume that A/a 2 - 2/a < 1/ c and denote 

<?> y)eM + :X - {whj) + y ' Ux-a^-a) ) + " > °} ' (5 - 3) 



~ 2a 2 ~ a ~ 2~c + 2a 2 (A - a) + 2a~o~ + \a ~ ~a? ' ' ' ' " 



which 



Let (<j)Q, ^i) 6 Do H (B^_ — Co). Then for any <f>i < <pi, (<^o, <j)\,<j)2) is in the stopping region of 

( Case ITb) Assume that A/a 2 — 2/a > 1/c (standing assumption in the rest of the theorem). Consider 
the four different possible ranges of parameters: 

Case II-b-i: If A + a < 

• and if —a/c — 1 < (Case II-b-i-1), then C in \3.5}) is the optimal continuation region for H3.3\ ). 

• Else if —a/c— 1 > ( Case II-b-i-2), then a superset of the continuation region can be constructed 
as follows. Let us introduce the line segment 

C^{(x,y)eM 2 + :x + y = X/c}, (5.5) 

and define 

C x 4 { (x, y)eM 2 + :x = x(t, x*),y = y(t, 0) for t e [0, t*] } 

(J \(x,y)eC:x< - „ 
^ I c X — a \ cl X — a \ c 

Here, t* is the solution ofy(-t*,-^ (l + §)) = and x* = x(-t*,^ + ^ (l + §)) The curve 
Ci separates R 2 ^ into two connected regions. Let us denote the region that lies above the curve 
Ci by 

Di = {(x,y) G B 2 ^ : there exists a positive number y(x) < y such that(x,y(x)) G C{\. (5.7) 
Then [(B 2 ^ — Bi) x R + ] n B^_ is an upper bound on the continuation region of H3.3\) . 
Case II-b-ii: If A + a > 

• and if —a/c — 1 < (Case II-b-ii-1), then C in US. 5\) is the optimal continuation region for \3.3\ ). 

• Else if -a/c - 1 > 0, then [(B 2 . - D x ) x R + ] fl B^_ is an upper bound on the continuation region 

of$M. 

Note that all the supersets of the continuation we constructed are bounded subsets o/R+. 
Proof. 
Note that 

®t > v(t, <M, >x(t,4>o), t>o, (5.8) 



2A / a\ 2A / a\] (5.6) 

1 + - , y > — r 1 + - 
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almost surely if $^ = fa and $q = fa. This is because $ x and $ + increase with jumps. 
From this observation we obtain the following inequality 



inf E„ 

res 



pt r pt 

/ e- 2Xs h(<S>*,<S>f)ds >infE / e - 2Xs h{x{s,fa),y(s,fa))ds 
Jo J re5 Uo 



(5.9) 



t 

2\s 



= inf / e- 2Xs h(x(s,fa),y(s,fa))ds. (5.10) 
te[o,oo] Jo 

Note that if for a given (fa, fa) the expression in (|5.10|) is equal to zero, then the infimum on the left 
hand side of (|5.9p is attained by setting r = 0. In what follows we will find a subset of the stopping 
region of the optimal stopping problem using this argument. 

Case Il-a: A/a 2 — 2/a < 1/c. In this case the mean reversion level of the path (x(-,fa),y(-,fa)), 
(fa, fa) £ M 2 ^, namely (A 2 /a 2 , — 2A/a), is inside the region Co which is defined in (|3.4[) . In this case, for 
any (fa, fa) £ B 2 ^ — Co the minimizer t op t(fa, fa) °f the expression in (|5. 10[) is either or oo by the 
following argument. For any (fa, fa) £ B+ — Co the path (x(-, fa), y(-, fa)) is in the advantageous region 
Co all the time except for possibly a finite duration. Therefore if 

e- 2Xs h{x{s, cf> ), y{s, fa))ds < 0, (5.11) 



o 

then in order to minimize ()5.10j) it is never optimal to stop. On the other hand if (|5.11j) is positive, then 
it is not worth taking the journey into the advantageous region and it is optimal to stop immediately in 
order to minimize (|5.10[) . 

We shall find the pairs (fa, fa) for which t opt — 0. Using 12 . 26[) we can write 

r 2Xs h(x(s, fa), y(s, fa))ds = 0o (-^g) + ^ ( (^Tffi ) + fc ' (5 ' 12) 

where k is given by (|5T|) . Note that if (fa, fa) G Do H (B^ - C ), then by (|53|) and (|5~T0f we can see 
that the infimum in (|5T0| is equal to 0. Therefore [((B^ - D ) U C ) x K + ] n B^_ is a superset of the 
optimal continuation region of 



Case Il-b: A/a 2 — 2/a > 1/c. In this case the mean reversion level of t — > (x(t, fa), y(t, fa)) is outside 
Co- Therefore, the minimizer of (|5. 10|) is t opt (^>o, 4>i) S {0, t c (fa, fa), °°} where t c (fa, fa) is the exit time 
of the path (x(t, fa) ,y(t, fa)) from Co- The derivative 

^-[x(t, fa) + y{t, fa)] = (A + a)y(t, fa) + 2ax(t, fa) + 2A (5.13) 
dt 

vanishes if (x(t, fa), y(t, fa)) meets the line segment 

L = {(x, y) £ M 2 + : (A + a)y + 2ax + 2A = 0}. (5.14) 
Note that the mean reversion level belongs to L, i.e., 

' A J,-^)ei. (5.15) 

Case II-b-i: A + a < 0. (In addition to a > (3, a < and A/a 2 — 2/a > 1/c.) In this case the line L is 
decreasing (as a function of x). 

Case II-b-i-1: —a/c - 2 < 0. (In addition to a > /3, a < 0, A/a 2 - 2/a > 1/c and A + a < 0.) In this 
case the line segment C in (15. 5p lies entirely below L. Assume that a path (a;(-, <^o)j y(-, fa)) originating 
at (fa, fa) £ M 2 , — Co enters Co at time to > 0. This path must leave Co at time t\ < oo since the mean 
reversion level (X 2 /a 2 ,—X/a) Co- This implies that for any t £ (to,h) x(t,fa) + y(t,fa) < A/c and 
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x(to, 0o) + y(to, 4>i) — V c ' This yields a contradiction, because A + a < together with (|5.13[) implies 
that t — > x(i, 0o ) + 0i ) is increasing below the line segment L. Therefore the minimizer i O pt(0(b 01 ) 
of (|5.10p is equal to if (0o,0i) ^ Co, and it is equal to i c (0o,0i) if (0 o ,0i) € Co- From (|5.9[) we can 
conclude that C is equal to the optimal continuation region of ()3.3|) . 

Case II-b-i-2: —a/c— 1 > 0. In this case the line segments C and L intersect at I = (x 1 ,y I ) = 
(| + (l + | ) , -^rj (l + § ))■ By running the paths backward in time, we can find x* such that 

(x*,0) = (x (-**,<) ,y H*,y 7 )) . (5.16) 
By the semi-group property (|2.27p , we have 

= a; (t* + (-«*),**) 

= a: (0, (5 7) 

Similarly, y(i*,0) = y 1 . The function t — > x(t 7 x*) + y(t, 0) is decreasing on (0,£*) and increasing on 
(t*,oo). It follows that the path t — > (x(t,x*),y(t, 0)) is tangential to C at 7 and lies above the region 
C . 

We will now show that if a path (x(-, 0o), y{-, 0i)) originates in Di, then it stays in Di. Let us first 
consider a pair (0o,0i) G Di such that 0i < — 2X/a. Consider the curve 

P = {(x,y) eB 2 + : x = x(t,x*),y = y(t,0) for t € [0,oo)} . (5.18) 

The following remark will be useful in completing the proof. 

Remark 5.1. The semi-group property in \2.2T\) implies that two distinct curves (x(-,4>q) , y{-, <fiD) 
and (x(-, 0q), y(-, <f)\)) do not intersect. If 

(x(t a ,r ), y(t a , 0?)) = (x(t b , 4), y(t\ 4)) = (0o, 0i) (5.19) 

for some t a ,t b <G E then (2JZ7\) implies that 

(x(t, 0g), y(t, 0?)) = (x(t a + (t- t a ),4),y(t a + (t- t a ), 0?)) 

- (x(t - t a , O ), y(t - t a , 00) = (x(t b + (t - t a ),4),y(t b + (t- t a ),4)) (5.20) 
= (x(t b -t a + t,cj) b Q ),y(t b -t a + t,<l> b 1 )), for all teR, 

i.e., the two curves are identical after a reparametrization. 

If the point (0o,0i) lies above P, and we recall that P lies above Co, then by Remark 15.11 the 
path (a;(-,0o),y(-,0i)) will lie above Co- If the point (0o,0i) lies between P and Co, then the path 
(x(-, 0o ),y(-, 0i )) will lie below the line segment L. This observation together with the fact that A + a < 
0, implies (using (JSTT3J)) that the function t — > x(t,4> ) + y(t,<t>i) is increasing. Therefore the path 
(x(-, 0o ),!/(-, 0i )) cannot intersect Co- 

Now let us consider a pair (0 o ,0i) € Di such that 0i > —2X/a. If (0o,0i) lies above L, then the 
function t — > x(t, O ) + y(t, 0i) is decreasing and its range is [0o + 0i, 2X/a(X/a — 1)), which is always 
above A/c, and therefore the path (x(-, 0o), y(-, 0i)) does not enter Co- If the point (0o, 0i) lies below L 
then t — > x{t, 0o)+j/(i, 0i) is increasing. This monotonicity implies that the path {x{-, 0o), y(-, 0i)) cannot 
visit Co- If 0i = — 2A/a, then y(t,(f>i) = — 2A/a for all t > 0. x(t,<po) increases or decreases depending 
on whether (0 O ,— 2A/a) is below or above L. Therefore if (0 O ,— 2A/a) ^ Co, then (x(-, O ), y(-, 0i)) never 
visits Co- These arguments show that if a path 0o), y(-, 0i)) originates in Di, then it stays in E>i. 
Therefore if (0o,0i) € Bi, then the infimum in (j^TO)) is equal to (by and ([STlO]) ). Therefore 

[(B+ — Oi) x R + ] n is a superset of the optimal continuation region of (|3.3[) . 

Case II-b-ii: X + a > 0. In this case L is increasing (as a function of x). The function t — > 0o) + 
y(i, 0i) is increasing if (0o, 0i) lies above L, and it is decreasing otherwise. 
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Case II-b-ii-1: —a/c — 1 < 0. In this case the line segments L and C do not intersect. Let us first 
consider a pair ((fro, <fri) G B^ such that cfri < — 2X/a. If ((fro, <j>i) ^ Co lies above the line segment L, then 
t — > x(t, 4>q) + y(t, 4>i) is increasing and the path (x(-, (fro), £/(•, (fri)) cannot enter Cq. Consider the curve 



P=\(x,y)G 



X 

t, — 
a 



,y = y(t,0) forte [0,oo) 



(5.21) 



which starts at the intersection of L with the x-axis. The semi-group property Remark l5.ll implies that 
no path starting to the right of P intersects P and therefore lies to the right of the region Co- Therefore, 
if (<po,(fri) is below the line segment L, then the path (x(-,(fro),y(-,(fri)) never visits the advantageous 
region Co- (Note that if the path (x(-, (fro), y(-, 4>i)) meets the line L at time ti,((fro,<fri), then t — > 
(x(t,(fr ) +y(t,(fri)) is decreasing (increasing) on [0,^] ([tL, oo).) 

Now let us consider a point (cfro, <fr\) S — Co such that <fri > — 2X/a. Then t — ► (x(t, (fro) + y(t, <fr{j) is 
increasing on [0, tL(<fro, <fri)] and is decreasing on (t^cfro, 0i), oo) (it decreases to — 2X/a + X 2 /a 2 > A/c). 
And the monotonicity of t — > x(t, (fro)+y(t, <pi) on [0, tL,(<fro, <fri)] implies that x(t, (fro)+y(t, <p\) > A/c for t S 
[0, t^icfro, <fri)\- If <fri = — 2X/a, then y(t, cfri) = — 2X/a for all t > 0. x(t, (fro) increases (decreases) depending 
on whether ((fro,— 2X/a) is above or below L. These arguments show that if a path (x(-,(fro),y(-,(frij) 
originates in B^ — Co, then it stays in B^_ — Co- Therefore the minimizer t op t((fro,(fri) of (|5.10[) for any 
(fro , (fri € B^_ — Co is equal to zero. Now using (|5.9p and (|5.10p the optimal continuation region of (|3.3p is 
equal to C. 

Case II-b-ii-2: — a/c— 1 > 0. In this case the line segments C and L intersect at / = (x 1 , y 1 ). Arguments 



similar to those of Case II-b-i-2 show that [(I 



_, in which Oi is defined in (|5.7j) . is a 



superset of the optimal continuation region of (|3.3[) . □ 

Theorem 5.2. Let us assume that a < [3 (Case III: a < (3) and define 

X + 2(3 



(5.22) 



Then 



$ 2 + - Di) x 
region of LS.3\) . 



n: 



B>2 = eB+ :x + y> 

which is a bounded region in M. 3 ,, is an upper bound on the continuation 



Proof. Note that in this case a > 0. The paths of the processes $ x , <I> + increase between the jumps 
and decrease with a jump. If r G S then there is a constant t > such that r A o~\ =t/\o~\ almost surely. 
Hence we can write 

fT A(Tl 



En 



-As 



h(^ s )ds 



Eo 



Eo 



e-* s h(^ s )ds 



e-* s h(^ s )ds 



En 



E 



l {r > CTl} f e- Xs h(V s )ds 

J <Ji 

3- Xs h(^ s )ds 



{t>ai} 



> 



thai 



e- Xs h(^ s )ds 



(5.23) 



- -Eo [l{t>*i}e- Xai ] 



2(3 

h(x(s, (fro), y(s, fa)) 



ds, 



using also the fact that o~\ has exponential distribution with rate 2(3. From (|5 . 23[) it follows that if 
x(s, (fro) + y(s, <fri) - (A + 2(3)/c > 0, then E [/J" e^ Xs h(^ s )ds] > for every stopping time r ^ 0, r e S. 
Since the paths x(t, (fro), y(t, <fr\) are increasing we can conclude that stopping immediately is optimal for 



That is r = is optimal for 



if 



and 



< 



in which D 2 is as in ([5T2"2"1) . □ 



Theorems 15.11 and 15.21 can be used to determine approximate detection rules besides helping us to 
determine the location and the shape of the continuation region. As we have seen in Cases II-b-i-1 and 
II-b-ii-1, these approximate rules turn out to be tight. The next theorem is essential in proving the fact 
that the smallest optimal stopping time of (|3.3p has a finite expectation. 



El 



Theorem 5.3 

A + 



Let to be the exit time of the process T from a bounded region D C 



Then 



[tjj] < oo for every 



- 1 ) G B?_. Hence t* defined in \5. 1\) has a finite expectation. 
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Proof. Let /(</> x ,</>+, 



Then it follows from (ET2"2"1) that 



[^/](0 X ,0+,0 1 ) = A0+ +2a0 x + 2A + a?!> + +/? 
(3 







- 1 







2_i 

/9 



(5.24) 



= 2A(</> X + + 1) > 2A, 



for every (0 X , </> + , cj) 1 ) G B+. Since / is bounded on D, and to A t is a bounded F-stopping time, we have 



Eo [/(T T . DAt )] = /(T )+E 



L4/](T s )cfe 



> 2XE [t d At] 



(5.25) 



On the other hand 



E [f(T TDM )}<-^ 

in which £ = min{a G R+ : for any (x, y, z) G D, max(x, y, z) < a} < oo. An application of the monotone 
convergence theorem implies that Eq[td] < oo. □ 

The results of this section can be used to determine approximate detection rules besides helping us to 
determine the location and the shape of the continuation region. As we have seen in Cases II-b-i-1 and 
II-b-ii-1, these approximate rules turn out to be tight. 

6. Optimal Stopping with Time Horizon a n . In this section, we will first approximate the 
optimal stopping problem (|3 - 3[) by a sequence of optimal stopping problems. Let us denote 



V n (a,b,c) = inf Eg' ' c 

tS5 



-At 



ft($ t *,*+)di 



(6.1) 



for all (a, b, c) € and n £ N. Here, a n is the n jump time of the process X 



Observe that (Vn)neN is a decreasing sequence and they each of its members satisfy — 1/c < V n < 0. 
Therefore the pointwise limit lim n V n exist. It can be shown that more is true using the fact that the 
function h is bounded from below and a n is a sum of independent exponential random variables. 



Lemma 6.1. For any (a,b,c) £ 



0<V n (a,b,c)-V(a,b,c) < 



23 



1 



c V2/3 + A 



(6.2) 



Proof. For any r G S, 



- xt h(<f>?,$t)dt 



dt 


= E 











e- xt h(*?,$+)dt 



En 



!{r>.„} / e- xt h(^,^)dt 



(6.3) 



The first term on the right-hand-side of (|6.3[) is greater than V n . Since /»(•,•) > -A/cwe can show that 
the second term is greater than 



hT>a n } I e Xs ds 



> _ Vo,<At r e -A CT „n > _l( 



(6.4) 



To show the last inequality we have used the fact that a n is a sum of n independent and identically 
distributed exponential random variables with rate 2/3. Now, the proof of the lemma follows immediately. 
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As in [1] and [B] to calculate the value functions V n iteratively we introduce the functional operators 
J, J t . These operators are defined through their actions on bounded functions g : — > K as follows: 



[Jg](t,a,b, c) = Ep' b ' c 



,-A< 



and, 



[ J t g] (a, b, c) = inf [Jg](s, a, b, c), te[0,oo]. 

s(Ht,oo 



(6.5) 



Observe that 

Eo [l {t > CTl} e- Affl 3 (^ 1 ,<i>+ ,^)] = E 
+ .9 



-1 l/« an 



/3 CTl "'/3 ai ' 



l)$i,_,^,_)l {x =^_ } )l {t > CTl} e-^ 



1 I z(s, c), — z(s, c) I ds 



= y*2(3e-^°g(^x(s,a),y(s,b)+^ 
+ 1 -J t 2Pe^ x+2 ^ s g (~ x(s, a), ~ y(s, b) - (j - l) z(s, c), z(s, c)\ ds 



(6.6) 



To derive f|6. 6|) we used the fact that <Ti has exponential distribution with rate 2/3, the dynamics in (|2.20[) . 
and the fact that conditioned on the event that there is a jump it has 1/2 probability of coming from X 1 
and 



Using (|6.6p and Fubini's theorem we can write 

[Jg](t, a, b,c)= f e-( x+2 ?>(h + /3 ■ g ° {F 1 + F 2 ))(x(s, a),y(s, b), z(s, c))ds 



where 



Using (|2.26[) it can be shown that 



i-l 



Fi(a,b,c) = -o, -3 b+ (-1)* — — 1 c, — 







/3 



2-i 



€{1,2}. 



(6.7) 



(6.8) 



lim [J#](t, a, 6, c) = [J#](oo, a, 6, c) < oo. 

t — *oo 



(6.9) 



Lemma 6.2. For every bounded function f, the mapping Jo/ is bounded. If f is a concave function, 
then Jof is also a concave function. If fx < fi, then Jo/i < Jo/2- 

Proof. The third assertion of the lemma directly follows from the representation (|6.7p . The first 
assertion holds since h is bounded from below and Jof (a, b, ) < J/(0, a, 6, c) = 0. The second assertion 
follows from the linearity of the functions x(t, •), y(t, •), h(-, •), •, •) and i^C'i •) •)• ^ 

Using Lemma 16.21 we can prove the following corollary: 
COROLLARY 6.1. Let us define a sequences of function (w„)„gN by 

v = 0, v n = J w«-i- (6.10) 

TTien every neN, u n is bounded and concave; and i>„+i < u„. Therefore, the limit v = lim„ ?; n exzsi, and 
is bounded and concave. Moreover, v n , for all n G N and i> are increasing in each of their arguments. 

Proof. The proof of the first part directly follows from Lemma 16.21 That v n for all n € N and v are 
increasing in each of their arguments follows from the fact that these functions are bounded from above 
and below and that they are concave. □ 
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Wc will need the following lemma to give a characterization of the stopping times of the filtration F 
(see 0). 

Lemma 6.3. For every t € S, there are T an measurable random variables : f2 — * oo such that 
t A (T n+ i = (a n + £„) A er rl +i Po almost surely on {r > a n }. The main theorem of this section can be 
proven by induction using Lemma 16.31 and the strong Markov property. 



Theorem 6.1. For every n S N, v n defined in Corollary 1 6. 1\ is equal to V n . For e > 0, let us denote 

r £ n (a,b,c) =M{te (0,oo] : [Jv n ](t, (a, b, c)) < [J Q v n ](a,b, c) + e}. (6.11) 

^4nd Zei us define a sequence of stopping times by Sf = rg(To) A o\ and 

Qe a J^ /2 (T ) if ax > r e n /2 (T ) 

I o~\ + bn o ai otherwise. 

Here 9 S is the shift operator on Q, i.e., X t o 6 S — X s+t - Then is an e-optimal stopping time of 
H6.1\). i.e., 



E 



a,b,c 



S 

At 



e- M h(V t )dt 



o 



< v n (a,b, c) +£, (6.13) 



in which * t = ($ t x , t>0. 
Proof. See Appendix. □ 



Theorem 16.11 shows that the value function V n of the optimal stopping problem defined in (16. lj) and 
the function v n introduced in Corollary 16.11 by an iterative application of the operator Jo are equal. 
This implies that the value function of the optimal stopping problem of (|6.1j) can be found by solving a 
sequence of deterministic minimization problems. 

7. Optimal Stopping Time. Theorem 7.1. r* defined in H5.1)) is the smallest optimal stopping 
time for (EM - 



This theorem shows that L defined in (|5.2|) is indeed an optimal stopping region. We will divide the 
proof of this theorem into several lemmas. 

The following dynamic programming principle can be proven by the special representation of the 
stopping times of a jump process (Lemma 16. 3|) and the strong Markov property. 

Lemma 7.1. For any bounded function g : — > K we have 

[J t g](a, b, c) - [Jg](t, a, b, c) + e -( A + 2 ^[Jo<?](x(t, a), y(t, b),z(t, c)). (7.1) 

Let us denote 

r n (a,b,c) =r°(a,6,c), (7.2) 

which is well defined because of (|6.9p and the continuity of the function t — > [Jf](t, a, b, c), t > 0. (See 
(|6.11[) for the definition of r°.) Let us also denote 

r(a, b, c) = inf{* > : [JV] (t, a, b, c) = J V(a, b, c)}. (7.3) 



COROLLARY 7.1. The functions r n and r defined by \7.2\j and J7. 3| ) respectively, satisfy 

r n (a, b, c) = {t > : v n+1 (x(t, a),y(t, b),z(t, c)) = 0} 
r(a, b, c) = {t > : v{x(t, a), y(t, b), z(t, c)) = 0}, 
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with the convention that inf '0 = 0. Together with j7.4\ ), Corollary \6.1\ implies that r n (a,b,c) | r(a,b,c) 
as n | oo. 

Proof. 

Suppose that r n (a 1 b,c) < oo. Then from (|6. 1 1|) it follows that 

[Jv n ](r n {a,b,c),a 7 b,c) = [J v n ](a, b,c) = [J rn(abc )V n }(a, b,c) 

(7 5) 

= [Jv n ](r n (a, b, c), a, b, c) + e" (A+2/3)r " (a ' f, ' c) u„+i(a;(r n (a, b, c), a), y{r n (a, b, c), 6), z(r n (a, b, c), c)). 

Here the second equality follows from the definition of the operator J t in (|6.5p and the third equation 
follows from Lemma 17. II and the fact that Jov n = v n +i- From (|7.5| it follows that v n +i(x(r n (a, b, c), a), 
y(r n (a,b, c),b),z(r n (a,b, c),c)) = 0. For t e (0,r„(o, 6, c)) we have [Jv n ](t,a,b,c) > [J v n ](a, b, c) 
— [^r„(o,6,c) u n]( a i c ) = Jtv n {a,b,c), since the function s — > [ J s « n ] (a, 6, c) is non-decreasing. Using 
Lemma 17.11 we can write 

[J v n ](a,b,c) = [J t v n ](a,b,c) = [Jv n }(t,a,b,c) + e-^ +2 ^v n+1 (x(t,a),y(t,b),z(t,c)), (7.6) 

which implies that v n +i(x(t, a), y(t, b), z(t, c)) < for all t < r n (a,b,c). 

Now, suppose that r n {a,b,c) — oo. Then v n+ i(x(t, a), y(t, b), z(t, c)) < for every t 6 (0,oo) which 
can be shown using same arguments as above. Therefore {t > : w n+ i(x(t, a), 6), z(t, c)) = 0} = 
and ([7T]) holds. 

The proof for the representation of r can be proven using the same line of argument and the fact that 
JqV = V. The fact that JqV — V can be proven by the dominated convergence theorem, since the 
sequences (v n (a, b, c)) n >o and ([Jv n ](t, a, b, c)) n >o are decreasing, and since v n is bounded function for 
all n € N. D 

In the next lemma we construct optimal stopping times for the family of problems introduced in (|6.ip . 

Lemma 7.2. Let us denote S n = S®, where is defined in Theorem \6.1\ for e > 0. Then the sequence 
(S l n ) ng N is an almost surely increasing sequence. Moreover S n < t* almost surely for all n. 

Proof. Since r% > 0, using Corollary 17.11 we can write 

{n - r , if ai > r x ) 

<ri-r +S 1 od ai , if r <a x <n \ > 0. (7.7) 
5*1 ° ^o-j if o-! <r J 

Now, let us assume that S n — S n —i > almost surely. From Lemma 17. II we have that r n > r n -\. Using 
this fact and the induction hypothesis we can write 



r n - r„_i, if (7i > r„ 

5„ + i - S„ = { ai - r„_i + 5„ o 6» CTl , if r n _i < cr x < r n ^ > 0, (7.; 
(S'n - S„_i) o 6» CTl if in < r„_i 

which proves the first assertion of the lemma. 

From Corollary 17.11 and the definition of r* it follows that r* A a\ — r A a x . Therefore t* A a% > 
fo A (7i = Si, since ro < r. Now, we will assume that S n < t* and show that S„+i < r*. On {cti < r„} 
we have that 

S n+ i =o~i+ S„ o ai < cti + r* o . (7.9) 

Since r* A o~\ — r A ci and r > r„, if cti < r n , then r'Adi = ci. Because r* is a hitting time, on the set 
{o~i < C {cti < r*} the following holds 

Sn+i < (71 + r* o ai = t* . 
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On the other hand if a\ > r n , then t* A a% = r A o\ > r n . Therefore on {a\ > r„}, S n +i — r n < t* . 
This concludes the proof of the second assertion. □ 

Lemma 7.3. Let us denote <3> t = t > 0. If S* = lining, then S* = r* almost surely. 

Moreover, r* is an optimal stopping time, i.e., 

V(a,b,c)=E%' b ' 




e- Xs h(^ s )ds 



Proof. The limit 5* = lim n S n exists since (SVi)ngN is increasing and S n < r* < oo (as a corollary of 
Theorem 15. 3p for all n. Let us show that S* is optimal. 



Eo 



lim / e~ xt h{^ t )dt 



< liminf Eo 



e- M h(^ t )dt 



(7.10) 



= lim V n (a, b, c) — V(a, b, c). 



The first inequality follows from Fatou's Lemma, which we can apply since 



e- M h(^ t )dt> / e- M h(^ t )dt > 



V2 



The first equality in (J7TTDJ) follows from Theorem 16.11 Now it can be seen from (|7.10[) that S* is an 
optimal stopping time. Taking the limit of (|6. 12[) as n — ► oo and using Corollary 17.11 we conclude that 

T* = S*. □ 

Proof of Theorem \7.1\ The proof of the optimality of r* follows directly from Lemma 17731 We will show 
that t* is the smallest optimal stopping time. 



Given any F-stopping time r < r* , let us define 



Then the stopping time f satisfies 



(7.11) 



e; 



a,b,c 




f e- Xs h(^ s )ds = E ! ' h ' c f e- Xs h(^ s )ds+ f e~ Xs h(y s )ds 

JO JO Jt 



= E 



a.b.c 



e~ Xs h(^ s )ds + e 



-At 



e Xs h(^ s+T )ds 



(7.12) 



= E 



a,b.c 



-As 



h(^ s )ds + e- XT V(T T ) 



< 



7 a.b,c 



e- Xs h{^ s )ds 



Here the third equality follows from the strong Markov property of the process T and the inequality 
follows since V(T T ) < 0. Equation (|7. 12[) shows that any optimal stopping time r < r* cannot be 
optimal. 

□ 

8. Structure of the Continuation and Stopping Regions Regions. Let us recall (15.2[) and 
denote 



r n ^{(a,b,c)eM 3 + :v n (a,b,c)^Q}, C„ 



r„. 



(8.1) 
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We have shown in Theorem 17.11 that that F of (|5.2p is the optimal stopping region for (|3.3[) and the 
first hitting time r* of T to this set is optimal. On the other hand although F n is an optimal stopping 
region for (|6.ip . the description of the optimal stopping times 5° (see (|6.12p ) is more involved. These 
optimal stopping times are not hitting times of the sets T n . S„ prescribes to stop if T hits T n before it 
jumps. Otherwise if there is a jump before T reaches T n , then S„ prescribes to stop when the process 
hits r„_i before the next jump, and so on. 

Theorem 16 . 1 1 shows that V n of (|6.ip and the functions v n introduced in Corollary [6J] are equal. There- 
fore, their respective limits V and v are also equal. Recall that V n converges to V uniformly and the 
convergence rate is exponential (see Lemma 16. 1[) . Since (w n ) n eN is a decreasing sequence with limit v the 
stopping regions in ()8.ip are nested and satisfy T C • • • C r„ C r„_i C ■••T\ and F = r\^ =1 F n . 

By Corollary 16 . 1 1 we know that each v n is concave and bounded, which also implies that the limit v is 
concave and bounded. This in turn implies that the stopping regions T n and F are convex and closed. 
Since we show in Section [5] that the continuation region is bounded, it can readily be shown that the 
stopping regions F n and F are the epigraphs of some mappings j n and 7 which are convex and strictly 
decreasing and the numbers x n = mf{y S M+ : 7n(y) = 0} and x = inf{y G R+ : j(x) = 0} are finite. 



9. Extensions. 



9.1. Non-identical Sources. Consider two independent Poisson processes X 1 and X 2 with arrival 
rates fa and fa respectively. At some random unobservable times B\ and 62, with distributions 

P(6' i = 0)=7r i F{9i > t) = (1 - vr 4 )e~ Al * for t > 0, (9.1) 

the arrival rates of the Poisson processes X 1 and X 2 change from fa to on, respectively, i.e., 



X l t - h % (s)ds, t>0,i=l,2, (9.2) 
Jo 

are martingales, in which 

hi{t) = \p i l {s< e i} + a i l {s > B . } ], t> 0,* = 1,2. (9.3) 

Here a\, 012, fa and fa are known positive constants. Then Equation the dynamics of $ x defined in 
(I2.19P becomes 

d$ x = [Aa^ 1 + Ai$? + ( 0l + a 2 )$t]dt + $ t x [{{ax/ fa) - l)dX} + {{a 2 /fa) ~ l)dX 2 ] , (9.4) 
in which a = Xi — a,: + fa, i € {1, 2}. Let us introduce 

x(t, <fo) = e (ai+Q2) Vo + f e^ +a ^-^ (X 2 y(u, (f>x) + X t z{u, <fc))du, in which 

(9.5) 
y(t, <j>i) = + e ait Ux + -) , z(t, <p 2 ) = -- + e Q2t (h + -) ■ 

Then $ x , and $ 2 , t > can be written as 

$ t x = x(t - a„, $ X J, $1 = y{t - a n , $+ ), $2 = z{ t _ CTnj $i j an <t< a n+1 , n e N, (9.6) 

and 



$ CT „+1 = ^7 1 {^„ +1 ^„ + 1 _} <1> ^ + 1)-' = ^ 1 W„ + 1 ^2„ +1 _ } $ -(n + l)- 



(9.7) 



Choosing Tt = , <3>t , f > as the Markovian statistic to work with, we can extend our analysis to deal 
with non-identical sources. 
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9.2. When there are more than two sources. We have solved a two-source quickest detection 
problem in which the aim is to detect the minimum of two disorder times. Our approach can easily be generalized 
to problems including several dimensions. To clarify how this generalization works, let us show what the sufficient 
statistics are when there are three independent sources. Assume that the observations come from the independent 
sources X , X 2 and X 3 . Let &t be the odds ratio defined in (|2.9[l . Then 

<S>t = *J + + + $1$? + + + (9.8) 

in which $\ i G {1,2,3} is defined as in (l2~14j) . Let us denote = i,j G {1,2,3} and = ^J* 2 *?, 

i > 0. The dynamics of these processes can be written as 

d^' j) = [A(*j + *j) + 2(A - a + 0)^}dt ^' 3) d{Xl + X/), 

V 7 (9.9) 
= [A (<&f ' 2) + $™ + $f 3 >) + 3(A - a + /9)*(»>] <tt + (| - l) ^dp^ + X? + Xf). 



We can see from (pT3|) and fl9U) that T = (<f>\ $ 2 , $ 3 , $f ' 2) , $f ' 3) , *f * 3) , $ w ) is a 7 dimensional Markovian 
statistic whose natural filtration is equal to the filtration generated by X 1 , X 2 and X 3 . From this one can see 
that the results of Sections [6] and [7] can be extended to the three dimensional case since these results rely only on 
the fact that the sufficient statistic T is a strong Markov process. The boundedness of the continuation region can 
also be shown as in Section \5\ since these results can be derived from the sample path properties of the sufficient 
statistic. 

As a result, our results are applicable for decision making with large-scale distributed networks of information 
sources. In the future, using the techniques developed here, we would like to solve a multi-source detection 
problem where the observations come from correlated sources. We also would like to extend our results and 
develop change detection algorithms that can be applied effectively to multiple source data that involves both 
continuous and discrete event phenomena. 

9.3. When the jump size of the observations are random. Consider two independent compound 
Poisson processes X 1 — {X\ : t > 0}, i G {1, 2}, where 

Xl^X^+Y.Yj, (9.10) 

3=1 

in which N z , i G {1,2} are two independent Poisson processes whose common rate /3 > changes to a at some 
random unobservable times 6i, i G {1, 2}, respectively. The random variables G K d , i G {1, 2}, which are also 
termed as 'marks', are independent and identically distributed with a common distribution, u, which is called 
as the 'mark distribution'. At the change time 9i the mark distribution of the process X r changes from v to fi. 
We will assume that /i is absolutely continuous with respect to v and denote the Radon-Nikodym derivative by 
r(y) = fj(y), V G Rd - In tnis case L \ in (E3J becomes 

N't , . 

The likelihood ratio process L % is the unique solution of the stochastic differential equation (see e.g. [8]) 

dLi = Li(-(a-P)dt + J (jjr(y)-l)p(dtdy)y 4 = 1, (9.12) 
where p is a random measure that is defined as 

oo 

p'((0,(]xi)^l {j ,< t) l {v , ej(}l t>0, (9.13) 
fc=i 

and for any A that is a Borel measurable subset of R d . Here a l k is the fc th jump time of the process X % . Now 
using the change of variable formula for semi-martingales (see e.g. [12]), we can write 

d^ = (A + (A-o + ^)*i)« + *«- / (^r{y)-l]p\dtdy), *J = (9.14) 

J ye m d VP / 1 -7Tj 



Quickest Detection of a Minimum of Two Poisson Disorder Times 



19 



for t > 0, and i £ {1, 2}. Note that = |f r / 'n)^a- n - a ^ the n th j um P time of the process X 1 . Using a change 
of variable formula for semi-martingales, the dynamics of $ x and in (|2.19[) can be written as 



[\<t>+ +a$?}dt + $t 



j/61 



(9.15) 



■r(y)-l) (p 1 +p 2 )(dtd2/), 
d$+ = [2A + a$+]dt + / f2 r (y) - l) p\dtdy) + / (- r(y) - l) p 2 (dtdy) 

JyeVL d \P J Jy£M d \P / 

with initial conditions $q = 7Ti7T2/[(l — i"i)(l — ^2)], and $J = ""i/C^ — ""l) + 7r 2/(l — ^2)- 

The bounds on the continuation region constructed for the simple Poisson disorder problem in Section [5] can 
also be shown to bound the continuation region of the compound Poisson disorder problem. On the other hand 
the results in Sections [6] and [7] can be shown to hold. The only change will be the form of the operator J in (|6.7[1 . 
But this new operator can be shown to share the same properties as its counterpart for the un-marked case. 
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10. Appendix. Proof of Theorem l6.ll We will prove only that V n = v n and is an e-optimal stopping 
time of EH. 



The proof will be carried out in three steps, 
(i) First we will show that V n > v„. To establish this fact, it is enough to show that for any stopping time r 6 S 



In order to prove (|10.ip we will show that 



/" 

Jo 



e~ xt h{V t )dt 



> v n (a,b,c). 



> Eo 









" e- xt h(V t )dt + l {T >^ k+1 }e- X,Tn - k+1 v k -i(r, n _ k+1 ) 



(10.1) 



(10.2) 



for fce{l,2,...,n + l}. Note that 1)10. 1|) follows from (|10.2|l if we set k = n + 1. In what follows we will to show 
(|10.2|) by induction. 

When k = 1, (|10.2p is satisfied since vo = 0. Assume that (|10.2p holds for 1 < k < n + 1. Let us denote the 
right-hand-side of (|10.2|) by pk-i- We can write pk-i = pl-i + Pk-i> where 



1 A m 

p fc _i = E 



'/i(*t)di 



and 



Pk 



-i=E 



l {T><r n -k} 



(10.3) 



Now by Lemma 16.31 there exists an Ta n _ k -measurable random variable £ n -fc such that 

T A <7 n _fc+i = (<r n -k + Cn-fc) A <J n -k+i almost surely on {r > cr„_&}. (10.4) 
Equation (|10.4p together with the strong Markov property of T (with respect to the filtration F) implies that 

pl-i =E [l {r > CTn _ fc} e~ ACT "- fc /fc-i(£n-fc,T CTn _ fc )J ; (10.5) 

in which 

' a \- xt h{^ t )dt + l {r >^e- x ^v k - X {T^) 



f k -i(r, (a, b, c)) = Ep 1 



A_ TT7<a,b,c 



(10.6) 



Jv k -\{r, (a,fe,c)) > JoVk-i(a,b,c) = v k (a,b,c), 



in which the second equality and the first inequality follow from (|6.5|1 and the last equality follows from (16. 10[l . 
Therefore 



Pk-i > Eo 1{ T > 



— AtT- _ U 



(T.„_ fc )] 



(10.7) 
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Now using (|10.2[) . (|10.3p and (|10.7|) we obtain that (|10.2|) holds when k is replaced by k + 1. At this point we 
have proved by induction that (|10.2[) holds for k = 1, 2, n + 1. 



(ii) The converse of (i), V n < v n , follows from (|6.13[) . since < a n by construction (see (|6.12[) ). 

(iii) What is left to prove is (|6.13|) . If n — 1, then the left-hand-side of (|6.13p becomes 

f J-Q (a,b,c) Acri 



K 



i: 



e - xt h(Vt)dt 



= Jv (ro(a, b, c),a,b,c) 
< JoVo{a, b,c) + e = vi(a, b, c) + e. 



(10.8) 



Now, suppose that (|6.13[) holds for all e > for some n. Using the fact that S^ +1 A o\ — r s J 2 A <ti almost surely 
and the strong Markov property of T, we can write 



E„ 



e- xt h{^t)dt 



= E 


[L 







3 ri+l AO "l 



in which 



e~ xt h(^t)dt 



g n (a,b,c) = Eq'' 



Jo 



e- xt h{<S> t )dt+l {SUi > ai} / e- xt h(^ t )dt 

'2 

e~ xt h(y t )dt < v n {a, b, c) + e/2. 



(10.9) 



(10.10) 



The inequality in (|10.10[l follows from the induction hypothesis. Using (|10.10p we can write (|10.9p as 

rS, 

o Uo 



e; 



a,b,c 



e- xt h(t> t )dt 



< E, 



a,b,c 



e- Xt h(^)dt+l {r ^ (abc) ^ i} e- x ^v n (T^) 



+ e/2 



= Jv n (r £ J 2 (a,b,c),a,b,c) +e/2 < v n+1 (a,b,c) + e. 
This proves (|6,13[l when n is replaced by n + 1. 



(10.11) 
□ 
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